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The  Honorable  Robert  C.  Byrd 
Ranking  Minority  Member 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Carl  Levin 
Ranking  Minority  Member 
Committee  on  Armed  Services 
United  States  Senate 

Since  the  Persian  Gulf  War,  Members  of  Congress  have  raised  concerns 
regarding  the  adequacy  of  technology  used  by  the  Department  of  Defense 
(DOD)  to  detect,  identify,  prepare  for,  and  protect  troops  against  chemical 
and  biological  (CB)  weapons.'  In  1993,  the  National  Defense  Authorization 
Act  for  Fiscal  Year  1994  (P.L.  103-160)  directed  the  Secretary  of  Defense  to 
take  actions  designed  to  improve  the  Department’s  CB  defense  capabilities, 
including  coordination  and  integration  of  all  CB  defense  programs  into 
what  is  now  the  CB  Defense  Program.  More  recentfy,  concerns  that 
terrorists  might  move  beyond  using  conventional  weapons  to  chemical  or 
biological  devices  led  Congress  to  authorize  the  federal  government  to 
improve  domestic  capabilities  to  respond  to  such  incidents.  With  the 
initiation  of  these  domestic  preparedness  programs  in  fiscal  year  1997, 
federal  research  and  development  efforts  to  develop  nonmedical 
CB  defense  technology  expanded  considerably,  and  they  continue  to  grow.^ 
According  to  the  White  House,  the  President’s  fiscal  year  2000  budget 
request  includes  over  $10  billion  to  combat  terrorism.  Almost  $1.4  billion  is 
for  programs  specifically  aimed  at  terrorist  threats  from  chemical, 
biological,  radiological,  or  nuclear  weapons,  an  amount  which  exceeds  the 


‘  See  Chemical  and  BloloBlcal  Defense:  Emphasis  Remains  Insufficient  to  Resolve  Continuing  Problems 
(GAO/NSIAD-96-103,  Mar.  29, 1996)  and  Chemical  Weapons:  DOD  Does  Not  Have  a  Strategy  to  Address 
Low-Level  Exposures  (GAO/NSIAD-98-228,  Sept  23,  1998). 

^  Nonmedical  technologies  refer  to  technologies  for  detecting,  identifying,  protecting  against  or 
decontaminating  personnel  and  equipment  of  chemical  and  biological  agents.  By  contrast,  examples  of 
medical  research  and  development  include  the  development  of  prophylactics  such  as  vaccines,  medical 
diagnostics  for  determining  exposure  to  chemical  or  biological  agents,  and  therapeutic  drugs  or 
procedures  for  countering  the  effects  of  exposure. 
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Results  in  Brief 


funding  of  less  than  $1  billion  for  military  programs  to  counter  chemical 
and  biological  threats. 

In  1993  Congress  enacted  the  Government  Performance  and  Results  Act 
(commonly  referred  to  as  the  Results  Act).  The  legislation  was  designed  to 
have  agencies  focus  on  the  performance  and  results  of  their  programs 
rather  than  on  program  activities  and  resources,  as  they  had  traditionally 
done.  Congress  sought  to  shift  federal  management  and  oversight  from  its 
preoccupation  with  program  staffing,  activity  levels,  and  tasks  completed 
to  program  results— that  is,  to  the  real  difference  that  federal  programs 
make  in  people’s  lives.  Congressional  reports  and  administrative  guidance 
indicate  that  programs  such  as  the  CB  Defense  Program  should  follow  the 
Results  Act’s  outcome-oriented  principles,  including  the  establishment  of 
general  goals  as  well  as  quantifiable,  measurable,  outcome-oriented 
performance  goals  and  related  measures. 

As  you  requested,  we  examined  the  extent  to  which  DOD  has  applied  the 
Results  Act’s  outcome-oriented  principles  to  the  CB  Defense  Program, 
focusing  in  particular  on  research,  development,  testing,  and  evaluation 
(RDT&E)  activities  that  lead  to  new  technologies  and  defensive 
capabilities.  Specifically,  we  assessed  whether  (1)  CB  Defense  Program 
goals  are  explicit  and  measurable,  (2)  the  CB  Defense  Program  has 
performance  measures  that  assess  outcomes  and  impacts  rather  than 
outputs  and  activities,  and  (3)  organizations  executing  the  CB  Defense 
RDT&E  activities  have  incorporated  Results  Act  principles  in  their  program 
planning  and  evaluation.  A  companion  report  Chemical  and  Rinlngiral 
Defense:  Coordination  of  Nonmedical  Chemical  and  Biological  Research 
and  Development  Programs  (GAO/NSIAD-99-160,  Aug.  16, 1999)  examines 
coordination  on  nonmedical  CB  defense  research  and  development 
programs. 


DOD’s  CB  Defense  Program  in  general,  and  its  RDT&E  activities  in 
particular,  have  not  incorporated  key  Results  Act  principles,  as  evidenced 
by  the  fact  that  the  goals  of  the  program  are  vague  and  unmeasurable  and 
do  not  articulate  specific  desired  impacts.  Program  planners  do  not 
explain,  for  example,  the  meaning  of  goals  such  as  denying  military 
advantage  or  allowing  U.S.  forces  to  operate  largely  unimpeded  by 
chemical  and  biological  attacks.  In  the  absence  of  explicit  and  measurable 
goals,  it  is  difficult  to  assess  whether  the  program  has  been  successful  in 
achieving  its  goals. 
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The  performance  measures  of  CB  Defense  Program  RDT&E  emphasize 
activities  rather  than  impacts.  The  program  is  not  being  evaluated 
according  to  its  impact  on  the  defensive  or  operational  capabilities  of 
U.S.  forces,  either  individually  or  collectively.  CB  Defense  Program 
planners  use  roadmaps  to  track  program  progress  toward  meeting 
chemical  and  biological  defense  goals.  These  goals  frequently  take  the 
form  of  advanced  concept  technology  demonstrations.^  However,  the 
demonstration  of  a  new  defensive  technology  or  capability  is  not  a  measure 
of  the  program’s  impact  or  contribution  to  the  military’s  ability  to  survive, 
fight,  and  win  in  chemical  and  biological  environments.  For  example,  these 
technology  demonstrations  may  still  need  additional  engineering  and 
manufacturing  development  or  product  and  concept  development,  as  well 
as  successful  operational  testing,  before  production  begins  and  warfighters 
are  equipped. 

CB  Defense  Program  research  and  development  organizations  have 
incorporated  Results  Act  principles  inconsistently.  Only  one  organization 
has  adopted  the  Results  Act  planning  and  evaluation  tools.  The  remaining 
research  and  development  organizations  cited  either  the  utilization  of 
equivalent  planning  tools  or  the  unique  challenges  of  evaluating  research 
and  development  activities  as  reasons  why  they  had  not  or  could  not  adopt 
the  Results  Act  processes. 

We  are  recommending  that  the  Secretary  of  Defense  take  actions  to 
develop  a  performance  plan  for  the  CB  Defense  Program  based  on  the 
outcome-oriented  management  principles  embodied  in  the  Results  Act. 


Background 


Consistent  with  the  National  Defense  Authorization  Act  for  Fiscal  Year 
1994,^  the  Secretary  of  Defense  assigned  responsibility  for  the  overall 
coordination  and  integration  of  the  CB  Defense  Program  to  a  single  office, 
the  Deputy  Assistant  to  the  Secretary  of  Defense  for  Counterproliferation 
and  Chemical/Biological  Defense.  The  office  is  responsible  for  approving 
all  planning,  programming,  and  budgeting  documents,  ensuring 


^  Advanced  concept  technology  demonstrations  assess  the  military  utility  of  mature  technologies  and 
their  capabilities  in  realistic  operational  scenarios.  CB  defense  capabilities  that  have  been  explored 
through  these  technology  demonstrations  include  the  capability  to  (1)  provide  early  warning  of  remote 
biological  warfare  agents;  (2)  detect,  warn,  dewarn,  identify,  protect,  and  decontaminate  air  bases  and 
seaports  against  biological  attack;  and  (3)  integrate  biological  and  chemical  detection  and  early 
warning  capability  at  an  air  base  or  seaport. 

^RL.  103-160,  sec.  1701. 
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coordination  between  the  medical  and  nonmedical  CB  defense  efforts,  and 
overseeing  management  oversight  in  accordance  with  the  law.  Several 
organizations  within  DOD  conduct  RDT&E  activities  on  behalf  of  the  CB 
Defense  Program.  These  include  the  Defense  Advanced  Research  Projects 
Agency  (DARPA),  the  Soldier  and  Biological  Chemical  Command,  and  the 
Joint  Program  Office  for  Biological  Defense.  In  addition,  CB  defense 
research  and  development  is  conducted  at  the  national  laboratories  of  the 
Department  of  Energy  (DOE),  funded  both  by  DOD  and  DOE. 

The  program  addresses  five  defensive  capabilities,  three  of  which  are 
nonmedical:  contamination  avoidance,  protection,  and  decontamination,® 
as  well  as  medical  chemical  defense  and  medical  biological  defense.  These 
areas  comprise  the  framework  that  DOD  uses  to  formulate  nonmedical  CB 
Defense  Program  requirements.  When  doctrinal,  training,  or  organizational 
solutions  cannot  satisfy  warfighters’  needs  in  these  areas,  DOD  seeks  new 
equipment  through  the  research,  development,  and  acquisition  cycle.®  CB 
defense  funding  is  divided  between  the  program’s  two  primary  activities: 
RDT&E  and  procurement.  Of  the  CB  Defense  Program  budget  of 
$717  million  proposed  for  fiscal  year  2000,  $340  million  (47  percent)  would 
be  for  RDT&E  and  the  remaining  $377  million  (53  percent)  for 
procurement. 

The  Results  Act  is  the  primaiy  legislative  framework  through  which 
agencies,  at  all  levels,  are  required  to  set  strategic  goals,  measure 
performance,  and  report  on  the  degree  to  which  goals  are  met.  The 
outcome-oriented  principles  of  the  Results  Act,  which  Congress 
anticipated  would  be  institutionalized  and  practiced  at  all  organizational 
levels  in  federal  agencies,  include  (1)  establishing  general  goals  and 
quantifiable,  measurable,  outcome-oriented  performance  goals  and  related 
measures;  (2)  developing  strategies  for  achieving  the  goals,  including 
strategies  for  overcoming  or  mitigating  major  impediments  to  goal 
achievement:  (3)  ensuring  that  goals  at  lower  organizational  levels  align 
with  and  support  general  goals;  and  (4)  identifying  the  resources  that  will 
be  required  to  achieve  the  goals. 


®  Contamination  avoidance  includes  detecting,  avoiding,  and  bypassing  contaminated  areas;  protection 
consists  of  individual  and  collective  protection;  decontamination  is  the  restoration  of  combat  power 
after  a  CB  attack. 

®  DOD  categorizes  RDT&E  into  five  budget  activities:  basic  research  (6.1  account),  applied  research 
(6.2),  advanced  technology  development  (6.3),  demonstration/validation  (6.4),  and  engineering  and 
manufacturing  development  (6.5).  DOD  refers  to  “activities  6.1  to  6.3"  as  Defense  Science  and 
Technology. 
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In  its  guidance  on  Results  Act  implementation,  the  Chief  Financial  Officers 
Council  advised  agencies  that  to  comply  with  the  spirit  and  intent  of  the 
act,  the  goals  and  measures  used  at  lower  organizational  levels  should  be 
linked  with  the  agency’s  strategic  goals. 

The  Quadrennial  Defense  Review  (QDR)  serves  as  DOD’s  overall  strategic 
planning  document.  The  QDR  directs  DOD  organizations  at  all  levels  to 
review  their  strategic  and  mission  objectives  in  order  to  ensure  that  they 
link  to  the  goals  and  objectives  of  the  QDR  and  that  Results  Act 
performance  plans  indicate  progress  toward  meeting  QDR  goals.  DOD 
implementing  guidance  states  that  the  goals,  objectives,  measures  of 
success,  quantifiable  performance  measures,  and  program  outcome 
evaluations  of  subordinate  organizations  should  be  linked  to  the  DOD 
corporate  goals  articulated  in  the  QDR  and  made  operational  in  the  DOD 
performance  plan.  DOD  does  not  routinely  link  its  performance  measures 
to  specific  organizational  units  or  individuals,  which  have  sufficient 
flexibility,  discretion,  and  authority  to  accomplish  desired  results.  DOD’s 
performance  plan  for  fiscal  year  2000  does  not  specifically  discuss  the  CB 
Defense  Program.  (CB  Defense  Program  RDT&E  activities  are  aggregated 
with  those  of  other  modernization  activities  to  support  DOD’s  second 
corporate  goal  to  “prepare  now  for  an  uncertain  future  by  pursuing  a 
focused  modernization  effort  that  maintains  U.S.  qualitative  superiority  in 
key  warfighting  capabilities.’’) 

Congressional  reports  and  administrative  guidance  indicate  that  programs 
such  as  the  CB  Defense  Program  should  follow  the  outcome-oriented 
principles  of  the  Results  Act.  In  our  assessment  of  the  adherence  of  the 
program  to  the  act,  we  only  evaluate  of  the  first  of  four  principles — 
establishing  general  goals  and  quantifiable,  measurable,  outcome-oriented 
performance  goals,  and  related  measures.^ 


^  Tasks  required  for  implementing  the  first  principle  include  (1)  identifying  the  organization’s  mission 
and  long-term  strategic  goals,  (2)  describing  how  the  organization’s  annual  performance  goals  are 
related  to  its  long-term  goals,  (3)  specifying  annual  performance  goals  for  each  program  activity, 

(4)  identifying  the  performance  measures  the  organization  will  use  to  assess  its  progress,  and 

(5)  describing  how  data  will  be  verified  and  validated. 
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Goals  of  the  CB 
Defense  Program  Are 
Vague  and 

Unmeasurable  and  Do 
Not  Articulate  Specific 
Desired  Impacts 


Although  DOD  has  taken  the  initial  and  necessary  step  of  articulating  CB 
Defense  Program  goals,  the  goals  are  not  articulated  in  a  manner  consistent 
with  Results  Act  principles.  The  stated  goals  are  vague  and  unmeasurable, 
and  they  fail  to  articulate  specific  desired  impacts.  A  Results  Act 
framework  requires  that  managers  define  a  related  set  of  long-term 
strategic  goals,  annual  agency  goals,  and  measurable  performance  goals  for 
each  program.  The  five  CB  Defense  Program  goals  are  to 

1.  deter  CB  weapon  use  by  denying  military  advantage  to  an  enemy 
through  a  combination  of  avoidance,  protection,  decontamination,  and 
medical  support  capabilities,  allowing  U.S.  forces  to  operate  largely 
unimpeded  by  chemical  and  biological  attacks  and  their  subsequent 
effects: 


2.  address  the  most  probable  CB  weapon  threats  that  could  be 
encountered  in  regional  conflicts  and  field  capabilities  to  the  forces 
required  for  two  major  theater  wars; 

3.  ensure  the  CB  weapon  Threat  Evaluation  Projection  drives  CB  defense 
research,  development,  and  acquisition  programs: 

4.  emphasize  a  joint  service  approach  to  CB  defense  research  and 
development,  and  acquisition;  and 

5.  complete  critical  RDT&E  and  acquisition  of  improved  CB  detection, 
identification,  and  warning  systems;  individual  and  collective  protection 
systems:  and  medical  support  and  decontamination  systems. 

Measuring  the  first  goal  is  unachievable,  determining  a  deterrence  effect  is 
problematic,  and  attributing  the  specific  rationale  for  the  deterrence  is 
unrealistic.  The  second,  third,  and  fourth  goals  address  the  size,  focus,  and 
coordination  of  the  program— not  program  outcomes.  Together,  these 
goals  direct  that  the  program  be  sufficiently  large  to  address  the  needs 
resulting  from  two  major  theater  wars;  sufficiently  focused  to  address  the 
likely  validated  threats:  and  sufficiently  coordinated  to  capitalize  on 
efficiencies  and  other  benefits  of  joint  requirements  determination, 
research,  development,  and  acquisition.  The  objective  of  the  fifth  goal  is 
measurable  but  speaks  to  program  outputs  without  addressing  program 
outcomes  or  impacts  (such  as  decreased  defensive  vulnerabilities  or 
increased  operational  capabilities).  The  completion  of  RDT&E  or 
procurement  cannot  be  assumed  to  result  in  a  positive  impact  on  the 
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defensive  posture  or  operational  flexibility  of  U.S.  forces.  While  the 
completion  of  these  activities  may  well  generate  benefits  for  U.S.  troops,  in 
the  absence  of  valid,  reliable  measures,  the  contributions  of  RDT&E  or 
procurement  cannot  be  determined. 


CB  Defense  Program 
Performance  Measures 
Emphasize  Activities 
Rather  Than  Outcomes 
and  Impacts 


The  CB  Defense  Program  is  not  evaluated  on  the  Impact  of  its  activities  on 
the  defensive  or  operational  capabilities  of  U.S.  forces,  either  individually 
or  collectively. 

CB  Defense  Program  planners  use  roadmaps.  Defense  Technology 
Objectives  (DTO)  and  advanced  concept  technology  demonstrations 
(ACTD)  to  assess  progress  toward  goals.  Program  planners  collectively 
prepare  a  number  of  strategic  plans  they  describe  as  “in  the  spirit  of  the 
Results  Act,”  if  not  specifically  for  the  purpose  of  assessing  outcomes  and 
impacts.  For  example,  DOD’s  Nuclear,  Biological,  and  Chemical  (NBC) 
Defense  Annual  Report  to  Congress*  and  the  Joint  Service  NBC  Defense 
Research,  Development,  and  Acquisition  Plan  are  updated  annually  and 
include  detailed  metrics  and  time  lines  reflecting  the  performance  of  the 
program  (such  as  the  demonstration  of  a  new  technology).  Roadmaps 
track  program  progress  toward  DTOs  that,  when  achieved,  DOD  claims  will 
create  new  operational  capabilities.  A  number  of  DTOs  are  ACTDs,  and 
plans  state  that  technology  demonstrations  provide  a  means  for  the  rapid 
field  testing  of  technical  options  to  solve  operational  needs.  CB  Defense 
Program  roadmaps  explicitly  link  the  completion  of  DTOs  and  ACTDs  with 
an  increase  in  the  demonstrated  warfighting  capabilities  of  U.S.  forces.  In 
addition,  CB  Defense  Program  planners  cited  ongoing  programmatic  peer 
reviews,  such  as  Technology  Area  Reviews  and  Assessments  (TARA),  as 
additional  means  to  measure  progress  toward  meeting  program  goals. 


We  do  not  concur  that  the  conduct  of  an  ACTD  or  a  peer  review  of  ongoing 
work  measures  the  impact  of  the  CB  Defense  Program  on  the  warfighter. 
Both  measures  have  limitations  that  make  them  inappropriate  for 
appraising  progress  toward  achieving  program  objectives.  ACTDs 
represent  a  means  for  rapidly  introducing  new  technologies  and  reducing 
the  time  from  the  start  of  a  program  to  the  system’s  initial  operational 
capability.  However,  the  demonstration  of  a  new  technology  may  not  by 
itself  result  in  the  effective  and  safe  deployment  of  a  military  capability  in 


*  Submitted  to  Congress  annually  pursuant  to  50  U.S.C.  1523. 
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support  of  the  warfighter.  Moreover,  as  we  previously  reported,  DOD  has 
not  always  emphasized  the  need  to  complete  concept  and  product 
development  or  testing  before  production,  thus  increasing  the  risk  of 
approving  ACTDs  in  support  of  CB  defense  that  include  immature 
technologies  and  then  prematurely  starting  production.® 


Technology  Area  Review 
Assessments  Are  Not  a 
Measure  of  Program  Impact 


TARAs  are  peer  reviews  conducted  by  the  Director,  Defense  Research  and 
Engineering  on  each  of  DOD’s  12  science  and  technology  programs— one 
being,  CB  defense.  TARAs  address  progress  toward  achieving  DTOs  and 
form  the  basis  of  the  Results  Act  rating  for  DOD’s  performance  in  science 
and  technology.*®  However,  the  application  of  the  TARA  to  generate 
performance  measures  of  DOD’s  science  and  technology  programs— such 
as  CB  defense — is  limited  by  several  factors.  First,  the  scope  of  the  TARA 
Results  Act  ratings  is  limited  because  TARAs  only  address  DTOs.  Funding 
for  DTOs  comprises  less  than  50  percent  of  total  funding  for  applied  and 
advanced  technology  research  and  development.  Thus,  the  Results  Act 
ratings  do  not  capture  the  majority  of  the  CB  Defense  Program’s  RDT&E 
activities.  Second,  the  focus  of  TARAs  is  on  budgets,  schedules,  and 
technical  performance.  TARAs  do  not  measure  technology  transition  from 
the  laboratory  to  the  battlefield.  Lastly,  TARAs  do  not  measure 
improvements  in  the  ability  of  U.S.  troops  to  survive,  fight,  and  win  in  a  CB 
environment. 


CB  Defense  Program 
Research  and 
Development 
Organizations  Have 
Incorporated  Results 
Act  Principles 
Inconsistently 


The  organizations  that  execute  or  contribute  to  the  research  and 
development  goals  of  the  CB  Defense  Program  vary  in  their  use  of  the 
Results  Act  principles  to  plan  and  assess  their  activities. 


°  Defense  Acquisition:  Advanced  Concept  Technology  Demonstration  Program  Can  Be  Improved 
(GAO/NSIAD-99-4.  Oct.  15, 1998). 

In  fiscal  year  1999,  there  were  23  chemical  and  biological  defense  DTOs,  many  in  the  form  of  ACTDs. 
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The  Soldier  and  Biological 
Chemical  Command  Is  the 
Only  RDT&E  Organization 
to  Systematically  Apply 
Results  Act  Principles 


The  Soldier  and  Biological  Chemical  Command  has  demonstrated  that  the 
Results  Act  principles  can  be  integrated  into  the  planhing  and  evaluation 
process  of  an  organization  conducting  research  and  development  for  the 
CB  Defense  Program.”  In  1996,  the  Chemical  and  Biological  Defense 
Command  (which  in  fiscal  year  1999  merged  with  the  Soldier  Systems 
Command  to  form  the  Soldier  and  Biological  Chemical  Command) 
developed  a  Business  Planning  Guide  aligned  with  the  Results  Act 
guidelines.  The  guide  linked  planning  guidance  of  higher  headquarters 
(the  Army  Materiel  Command):  the  Army’s  Planning,  Programming, 
Budgeting,  and  Execution  System:  and  the  Results  Act.  The  guide  stated 
that  the  Results  Act  “is  the  basis  for  the  three  products  that  result  from  our 
business  planning  process  outlined  in  this  document:  a  strategic  plan,  an 
annual  performance  plan,  and  an  annual  report.”  Subsequent^,  the 
Chemical  and  Biological  Defense  Command  issued  strategic  plans  for  the 
fiscal  years  1997-2003  and  fiscal  years  1998-2004  time  frames,  annual 
performance  plans  for  fiscal  years  1997  and  1998,  and  an  annual  report  for 
fiscal  year  1997. 


The  Chemical  and  Biological  Defense  Command  strategic  plem  is  driven  by 
and  linked  with  the  overarching  planning  architectures  of  DOD,  the  Army, 
and  the  Army  Materiel  Command.  The  Chemical  and  Biological  Defense 
Command’s  strategic  planning  model  directly  links  the  attainment  of  its 
vision  with  the  development  of  goals  and  enabling  strategies — followed  by 
the  execution  of  the  strategies  and  measurement  of  performance.  Separate 
measures  were  developed  to  assess  goal  achievement  as  well  as  progress 
toward  goal  achievement. 

Specifically,  the  plan  identified  Chemical  and  Biological  Defense  Command 
goals,  strategies  for  achieving  the  goals,  and  measures  of  goal  achievement. 
Two  of  the  Command’s  six  goals  address  their  potential  contributions  to  the 
CB  Defense  Program.  They  are  to 


•  research,  develop,  acquire,  and  field  NBC  defense,  smoke,  and 
obscurant  materiel  that  meets  warfighter  requirements  and  reduces 
acquisition  costs  and  timeliness  (i.e.,  produces  products  faster  and  at 
lower  costs)  and 

•  become  the  organization  of  choice  for  chemical,  biological,  and 
smoke/obscurant  research,  development,  and  technology  services. 


We  did  not  assess  the  comprehensiveness,  quality,  or  effectiveness  of  the  command’s  effort 
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The  performance  plan  identifies  performance  measures  for  each  Command 
goal  and  performance  goals  for  each  strategy.  The  performance  measures 
address  both  accomplishments  and  progress  toward  accomplishments. 
Examples  of  quantitative  measures  of  research  and  development 
accomplishments  include  (1)  the  percentage  of  new  NBC  systems  that 
meet  NBC  survivability  requirements,  (2)  the  percentage  of  nonexempt 
acquisitions  receiving  waivers  from  performance  specifications,  and  (3)  the 
percentage  of  Soldier  and  Biological  Chemical  Command  science  and 
technology  programs  transitioning  to  joint  service  and  Army  development 
programs  with  user  validation  through  modeling,  wargames,  or  similar 
methods. 

Soldier  and  Biological  Chemical  Command  officials  explained  that  the 
Command  is  still  evaluating  some  of  the  fundamental  dilemmas  in  applying 
Results  Act  principles.  For  example,  the  Command  has  yet  to  agree  on 
what  the  right  measures  are.  The  identification  of  measures  in  the  research 
and  development  component  of  the  Soldier  and  Biological  Chemical 
Command  (and  CB  defense  in  general)  has  been  an  ongoing  challenge  and 
continues  to  evolve. 


Other  CB  Defense  RDT&E 
Organizations  Have  Not 
Applied  Results  Act 
Principles 


The  remaining  RDT&E  organizations  cited  a  variety  of  reasons  for  not 
incorporating  the  Results  Act’s  principles  in  their  program  planning  or 
evaluation  systems.*^  The  two  most  common  reasons  cited  were  that 
current  DOD  planning  processes  were  equivalent  to  those  of  the  act, 
resulting  in  plans  that  were  “in  the  spirit”  of  the  Results  Act,  and  that  the 
unique  nature  of  RDT&E  activities  did  not  lend  itself  to  the  act’s 
performance  measurement  and  evaluation. 


CB  Defense  Program  Planning  DOD  officials  explained  that  the  Office  of  the  Secretary  of  Defense,  NBC 
and  Evaluation  Defense  Steering  Committee,  conducts  planning  consistent  with  Results 

Act  principles  for  the  CB  Defense  Program  by  issuing  and  executing  the 
Program  Objectives  Memorandum.  They  claimed  that 
memorandum-related  planning  documents,  including  reports  that  address 
different  aspects  of  the  CB  Defense  Program  (such  as  the  Joint  Operational 
Concept:  the  Research,  Development,  and  Acquisition  Plan;  the  Joint 
Logistics  Support  Plan;  and  the  Joint  Modernization  Plan)  as  well  as  the 


Of  the  remaining  principal  federal  organizations  that  conduct  RDT&E  on  CB  defense  topics,  only 
doe’s  CB  Nonproliferation  Program  has  developed  a  strategic  plan,  and  none  have  developed  a 
performance  plan. 
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Measurement  of  RDT&E 
Activities 


NBC  Defense  Annual  Report  to  Congress,  constitute  the  equivalent  of  a 
strategic  plan.  Moreover,  CB  Defense  Program  managers  stated  that  DOD’s 
Planning,  Programming,  and  Budgeting  System  is  equivalent  to  the  system 
required  by  the  act  and  that  therefore  no  substantive  changes  are  necessary 
to  comply  with  the  spirit  of  the  legislation.*® 

There  is  no  consensus  on  the  appropriateness  of  applying  performance 
measures  to  RDT&E  activities.  While  the  Soldier  and  Biological  Chemical 
Command  has  developed  and  applied  measures  of  research  and 
development  outcomes,  other  organizations  conducting  CB  Defense 
Program  RDT&E  have  not.  Neither  DARPA,  DOE,  nor  the  Joint  Program 
Office  for  Biological  Defense  have  taken  the  initiative  to  develop  a 
performance  plan.  In  its  strategic  plan,  DOE  included  a  5-year  roadmap  for 
developing,  demonstrating,  and  delivering  technology  that  would  lead  to 
major  improvements  in  preparedness  and  capabilities.  The  Joint  Program 
Office  cites  the  conduct  of  ACTDs  as  measures  of  its  performance.  DARPA 
officials  maintain  that  the  nature  of  their  RDT&E  activities  do  not  lend 
themselves  to  the  application  of  performance  measurement. 

DARPA  conducts  leading-edge  research  where  the  risks  of  failure  are  high 
and  the  probability  of  success  is  low.  Its  mission  is  to  pursue  long-term, 
far-reaching,  and  high-risk/high-payoff  technology  for  military  systems  in 
the  distant  future.  DARPA  officials  argued  that  developing  useable  metrics 
that  are  measurable,  relevant,  and  timely  for  technology  anticipated  10  or 
more  years  into  the  future  is  impossible.  Moreover,  they  stated  that  goals 
and  expectations  are  set  at  the  project  level  and  cannot  be  aggregated  at 
the  program  level.  Therefore,  according  to  the  officials,  it  would  be 
inappropriate  to  develop  programwide  or  agenc3wide  measures  of  success 
or  performance. 

Nonetheless,  DARPA  did  try  to  develop  a  performance  contract  and 
submitted  a  draft  to  the  Defense  Management  Council  in  early  1998.  It  also 
argued,  however,  that  it  did  not  fit  the  mold  of  most  DOD  agencies  and 
should  therefore  be  exempt  from  the  act’s  requirements.  DARPA  drafted 
performance  metrics  addressing  its  research  operations  as  well  as 
administrative  efficiency.  The  performance  metrics  proposed  for  the 
research  portion  of  its  activities  consisted  of  a  series  of  assessments 


The  DOD  Comptroller  has  noted  that  the  Results  Act  is  related  to,  but  distinct  from,  DOD’s  Planning, 
Programming,  and  Budgeting  System,  and  has  stated  that  Results  Act  planning  and  program  evaluations 
need  to  be  integrated  with  DOD’s  Planning,  Programming,  and  Budgeting  System. 
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performed  by  an  independent  panel  of  experts.'^  In  December  1998,  the 
Defense  Management  Council  notified  DARPA  that  it  was  exempt  from  the 
requirement.  The  draft  performance  contract  was  never  finalized  or 
implemented. 

Congress  has  recognized  that  successful  implementation  on  the  Results  Act 
in  science  agencies  would  not  come  quickly  or  easily.  Nonetheless, 
research  organizations  have  concluded  that  the  Results  Act  can  or  should 
be  applied.  The  Research  Roundtable,  a  group  of  federal  researchers  and 
managers  representing  a  cross  section  of  departments  and  agencies, 
concluded  in  1995  that  the  results  of  a  research  program’s  performance  can 
be  measured.  The  Army  Research  Laboratory  was  designated  as  a  pilot 
project  for  performance  measurement  under  the  act  and  ultimately 
outlined  an  evaluation  approach  that  made  use  of  three  pillars:  metrics, 
peer  review,  and  customer  feedback.  In  1999,  the  Committee  on  Science, 
Engineering,  and  Public  Policy  of  the  National  Academy  of  Sciences,  the 
National  Academy  of  Engineering,  and  the  Institute  of  Medicine  reported 
on  the  results  of  their  work  on  the  issue  of  measuring  and  evaluating 
research  in  compliance  with  the  act’s  requirements.  The  committee 
concluded  that  both  applied  research  and  basic  research  programs 
supported  by  the  federal  government  can  be  evaluated  meaningfully  on  a 
regular  basis. 


Conclusions 


The  CB  defense  research  and  development  outcomes  and  impacts  are  not 
being  systematically  measured.  The  CB  Defense  Program  lacks  both 
quantifiable  performance  measures  and  measurable  objectives.  In  the 
absence  of  measures  of  program  impacts  and  measurable  objectives, 
progress  toward  achieving  progreun  goals  cannot  be  determined.  Program 
planning  consists  of  a  series  of  roadmaps  leading  to  specific  equipment 
items.  Managers  cite  activity  measures  and  technology  demonstrations  as 
measures  of  the  program’s  contribution.  These  planning  and  programming 
steps  are  appropriate  and  necessary,  but  they  are  insufficient  for 
quantifying  outcomes  and  impacts.  Current  measures  do  not  assess  the 
incremental  changes  attributable,  in  whole  or  in  part,  to  the  CB  Defense 


The  Defense  Science  Board  was  the  proposed  panel  It  would  have  been  tasked  to  review  the 
portfolio  of  DARPA  projects  to  assess  projects  with  regard  to  (1)  relevance  to  warfighters,  (2)  ratio  of 
technology  investments  versus  system  development,  (3)  level  of  risk,  (4)  ratio  of  new  versus  continuing 
efforts,  and  (5)  level  of  service  and  commercial  sector  participation. 
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Program  that  improve  warfighters’  ability  to  survive,  fight,  and  win  in  a  CB 
environment. 

The  Results  Act  outcome-oriented  principles  have  not  been  widely  applied 
by  either  CB  Defense  Program  planners  or  executing  organizations.  The 
utilization  of  these  principles  can  enable  managers  and  those  overseeing 
the  program  to  quantify  the  relative  success  of  the  program  and  of 
component  projects  in  satisfying  requirements  across  different  activities 
(e.g.,  point  detection,  early  warning,  warning  and  reporting,  and  modeling) . 
Impact  measures  can  provide  a  planning  tool  to  allocate  finite  CB  Defense 
Program  resources  among  competing  sets  of  unmet  requirements. 


ReCOmniendationS  recommend  that  the  Secretary  of  Defense  take  actions  to  develop  a 

performance  plan  for  the  CB  Defense  Program  based  on  the 
outcome-oriented  management  principles  embodied  in  the  Results  Act. 
The  plan  should  be  agreed  to  and  supported  by  the  relevant  RDT&E 
organizations  and  incorporated  in  DOD’s  NBC  Defense  Annual  Report  to 
Congress.  Specifically,  the  plan  should 

•  establish  explicit  and  outcome-oriented  goals  linked  to  warfighters’ 
ability  to  survive,  fight,  and  win  in  a  CB  environment: 

•  identify  quantitative  or  qualitative  performance  measures  that  can  be 
used  to  assess  progress  toward  goal  achievement: 

•  describe  how  performance  data  would  be  validated: 

•  describe  how  RDT&E  activities  of  participating  DOD  and  non-DOD 
organizations  are  coordinated  to  achieve  program  goals:  and 

•  identify  human  capital,  financial,  and  resource  challenges  or  external 
factors  that  limit  the  ability  of  the  program  to  achieve  its  goals. 


Agency  Comments 


In  written  comments  on  a  draft  of  this  report,  DOD  concurred  with  our 
recommendation.  DOD  stated  it  will  develop  a  strategic  plan  more  closely 
aligned  with  the  tenets  of  the  Results  Act  and  publish  the  plan  in  the  next 
DOD  NBC  Defense  Annual  Report  to  Congress.  DOD’s  comments  are 
reprinted  in  appendix  I.  Our  scope  and  methodology  are  explained  in 
appendix  II,  and  CB  Defense  Program  RDT&E  organizations  are  described 
in  appendix  III. 
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As  agreed  with  your  offices,  unless  you  publicly  announce  its  contents 
earlier,  we  plan  no  further  distribution  of  this  report  until  7  days  after  its 
issue  date.  At  that  time,  we  will  send  copies  to  the  Honorable  William  S. 
Cohen,  Secretary  of  Defense  and  other  congressional  committees.  We  will 
also  make  copies  available  to  others  on  request. 

If  you  have  any  questions  regarding  this  report,  please  contact  me  or 
Sushil  K.  Sharma  at  (202)  512-3092.  Key  contributors  to  this  assignment 
were  Jeffrey  Harris  and  Weihsueh  Chiu. 


Kwai-Cheung  Chan 

Director,  Special  Studies  and  Evaluations 
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Abbreviations 


ACTD 

CB 

DARPA 

DOD 

DOE 

DTO 

NBC 

QDR 

RDT&E 

TARA 


Advanced  Concept  Technology  Demonstration 

chemical  and  biological 

Defense  Advanced  Research  Projects  Agency 

Department  of  Defense 

Department  of  Energy 

Defense  Technology  Objective 

Nuclear,  Biological,  and  Chemical 

Quadrennial  Defense  Review 

research,  development,  testing!  and  evaluation 

Technology  Area  Review  Assessment 
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Comments  From  the  Department  of  Defense 


NUCLEAR  AND  CHEMtCAL 
AND  BIOLOGtCAL  DEFENSE 
PROGRAMS 


ASSISTANT  TO  THE  SECRETARY  OF  DEFENSE 
3050  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3050 


Mr.  Kwai-Cheung  Chan  ^ 

Director,  Special  Studies  and  Evaluation 
National  Security  and  International  Affairs  Division 
U.S.  General  Accounting  Office 
Washington,  D.C.  20548 

Dear  Mr.  Chan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  General  Accounting 
Office  (GAO)  draft,  “CHEMICAL  AND  BIOLOGICAL  DEFENSE:  Program  Planning 
and  Evaluation  Should  Follow  Results  Act  Framework,"  dated  June  10, 1999  (GAO 
Code  71 3039/OSD  Case  1 839). 

The  DoD  concurs  with  the  draft  report  and  plans  to  develop  a  strategic  plan  that 
will  incorporate  tenets  of  the  Government  Performance  and  Results  Act  (GPRA). 

The  plan  will  be  coordinated  with  relevant  RDT&E  organizations  and  published  in  the 
next  version  of  the  DoD  Nuclear,  Biological  and  Chemical  Defense  Annual  Report  to 
Congress. 


1 V.  Wade,  Colonel,  U.S.  Army 
"Deputy  for  Chemical/Biological  Defense 
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General  Accounting  Office  Draft  Report 
“CHEMICAL  AND  BIOLOGICAL  DEFENSE: 
Program  Planning  and  Evaluation 
Should  Follow  Results  Act 
Framework,”  dated  June  10, 1999, 

(GAO  Code  713039/OSD  Case  1839) 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


Recommendation  1 :  The  GAO  recommended  that  the  Secretary  of  Defense  direct 
that  actions  be  taken  to  develop  a  performance  plan  for  the  Chemical  and  Biological 
Defense  Program  based  on  the  outcome  oriented  management  principles  embodied 
in  the  Results  Act.  The  plan  must  be  agreed  to  and  supported  by  the  relevant 
RDT&E  organizations  and  incorporated  in  DoD’s  Nuclear,  Biological,  and  Chemical 
Defense  Annual  Report  to  Congress.  Specifically,  the  plan  should: 

•  Establish  explicit  and  outcome-oriented  goals  linked  to  the  warfighter’s  ability  to 
survive,  fight  and  win  in  a  chemical  and  biological  environment, 

•  Identify  quantitative  or  qualitative  performance  measures  that  can  be  used  to 
assess  progress  toward  goal  achievement, 

•  Describe  how  performance  data  would  be  validated, 

•  Describe  how  RDT&E  activities  of  participating  DoD  and  non-DoD  organizations 
are  coordinated  to  achieve  program  goals,  and 

•  Identify  human,  financial,  and  resource  shortfalls  or  external  factors  that  limit  the 
ability  of  the  program  to  achieve  its  goals,  (page  13  &  14  of  Draft  Report) 

DoD  Response: 

The  DoD  concurs  with  the  draft  report  and  plans  to  develop  a  strategic  plan 
that  more  closely  aligns  itself  with  the  tenets  of  the  Government  Performance  and 
Results  Act  (GPRA)  requirements.  For  instance,  new  performance  measures  will 
address  capabilities  achieved  rather  than  just  numbers  of  systems  fielded.  A  full 
detailed  and  coordinated  plan  will  be  completed  and  published  In  the  next  version  of 
the  DoD  Nuclear,  Biological,  and  Chemical  Defense  Annual  Report  to  Congress. 
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Scope  and  Methodology 


The  scope  of  our  study  was  limited  to  the  nonmedical  research, 
development,  testing,  and  evaluation  (RDT&E)  activities  of  the  Department 
of  Defense’s  (DOD)Chemical  and  Biological  (CB)  Defense  Program.  This 
study  does  not  address  any  classified  programs  or  projects. 

To  determine  whether  the  program  goals  were  explicit  and  measurable  and 
whether  its  performance  measures  assessed  outcomes  and  impacts  rather 
than  outputs  and  activities,  we  reviewed  the  legislative  record,  interviewed 
agency  officials,  and  analyzed  program  documents.  To  understand  the 
requirements  of  the  Government  Performance  and  Results  Act,  we 
reviewed  the  legislation  as  well  as  implementation  guidance  issued  by  the 
Office  of  Management  and  Budget,  DOD  Comptroller,  and  the  General 
Accounting  Office.*  We  queried  representatives  of  the  Office  of  the 
Secretary  of  Defense’s  Nuclear,  Biological  and  Chemical  (NBC)  Defense 
Steering  Committee  and  the  executing  organizations  (i.e..  Joint  Program 
Office,  Soldier  and  Biological  Chemical  Command,  Defense  Advanced 
Research  Projects  Agency  (DARPA))  regarding  their  familiarity  with,  and 
utilization  of,  the  Results  Act’s  performance  measurement  principles.  We 
reviewed  strategic  and  performance  plans  implementing  the  act  as  well  as 
documents  characterized  by  DOD  officials  as  complying  with  the  “spirit  of 
the  legislation.”^ 

To  determine  whether  organizations  executing  the  CB  Defense  RDT&E 
activities  have  incorporated  Results  Act  principles  in  their  program 
planning  and  evaluation,  we  interviewed  program  officials,  examined 
program  documents,  observed  program  review  meetings,  and  attended 
scientific  conferences  on  CB  defense  technologies.  Program  documents 
we  examined  included  program  budgets,  strategic  and  performance  plans, 
annual  reports,  internal  program  planning  documents,  program  brlefmgs, 
and  proceedings  of  program  review  meetings.  We  also  observed  the  1999 
DOD  Technology  Area  Review  and  Assessment  of  chemical  and  biological 
defense.  Scientific  conferences  we  attended  included  the  1998  Joint 
Workshop  on  Standoff  Detection  for  Chemical  and  Biological  Defense  and 
the  1998  Scientific  Conference  on  Chemical  and  Biological  Defense 


'  See  Executive  Guide:  Effertively  Implementing  ths  Government  Performance  and  Rfl.siilts  Act 
(GAO/GGD-96-118,  June  1996);  Agencies'  Annual  Performance  Plans  Under  the  Results  Act:  An 
Asses.sment  Guide  to  Facilitate  Congressional  Decisionmaking  (GAO/GGD/AIMD-10.1.18.  February 
1998) :  and  The  Results  Act:  An  Evaluator's  Guide  to  Assessing  Agency  Annual  Performance  Plans 
(GAO/GGD-10.1.20,  April  1998). 

*  These  include  the  DOD  NBC  Defense  Annual  Report  to  Congress  (submitted  pursuant  to  50  CFR 
1523),  the  Joint  NBC  Modernization  Plan,  the  Joint  Service  NBC  Defense  Research,  Development,  and 
Acquisition  Plan,  and  the  Joint  Service  NBC  Defense  Logistics  Plan. 
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Research.  We  also  obtained  proceedings  from  these  and  other  scientific 
conferences. 

To  respond  to  all  three  objectives,  we  contacted  the  following 
organizations:  DARPA,  Arlington,  Virginia;  Defense  Threat  Reduction 
Agency,  Dulles,  Virginia:  DOD  Inspector  General,  Washington,  D.C.; 
Department  of  Energy,  Washington,  D.C.;  Director,  Defense  Research  and 
Engineering,  Washington,  D.C.;  Dugway  Proving  Ground,  Dugway,  Utah; 
Edgewood  Chemical  and  Biological  Center,  Aberdeen  Proving  Ground, 
Maryland:  Joint  Program  Office  for  Biological  Defense,  Falls  Church, 
Virginia;  National  Domestic  Preparedness  Office,  Washington,  D.C.; 
National  Ground  Intelligence  Center,  Charlottesville,  Virginia: 
Nonproliferation  and  National  Security  Office,  Department  of  Energy, 
Washington,  D.C.;  Office  of  Science  and  Technology  Policy,  White  House, 
Washington,  D.C.;  Office  of  the  Secretary  of  Defense,  Washington,  D.C.; 
U.S.  Army  Soldier  and  Biological  Chemical  Command,  Aberdeen  Proving 
Ground,  Maryland:  and  U.S.  Army  Training  and  Doctrine  Command,  Fort 
Monroe,  Virginia. 

We  conducted  our  review  from  November  1998  to  April  1999  in  accordance 
with  generally  accepted  government  auditing  standards. 


Page  21 


GAO/NSIAD-99-159  Chemical  and  Biological  Defense 


Appendix  III _ _ _ 

Chemical  and  Biological  Defense  Program 
Research,  Development,  Test,  and  Evaluation 
Organizations 


The  CB  Defense  Program  is  overseen  by  the  Office  of  the  Secretary  of 
Defense’s  Nuclear,  Biological,  and  Chemical  Defense  Steering  Committee, 
which  is  comprised  of  the  Directors  of  the  Defense  Threat  Reduction 
Agency  and  Defense  Research  and  Engineering  as  well  as  their  top  officials 
responsible  for  CB  defense.  The  steering  committee  funds  research  and 
development  at  numerous  laboratories  in  DOD,  Department  of  Energy,  and 
private  industry.* 

As  illustrated  in  figure  III.1,  key  research  and  development  organizations  in 
the  execution  of  the  program  include  the  SBCCOM,  JPO-BD,  DARPA. 


*  The  CB  Defense  Program  addresses  nonmedical  research  and  development  in  the  areas  of  chemical 
detection,  biological  detection,  individual  protection,  collective  protection,  decontamination,  modeling 
and  simulation,  core  science  and  technology,  and  basic  research.  In  addition,  core  science  and 
technology  includes  threat  assessment  and  aerosol  technology;  and  basic  research  includes  aerosol 
science,  chemistry  and  toxicology,  and  analytical  chemistry. 
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Chemical  and  Biological  Defense  Program 
Research,  Development,  Test,  and  Evaluation 
Organizations 


Figure  III.1:  Primary  Planning  and  Executing  Organizations  and  Programs  of  DOD’s  Chemical  and  Biological  Defense  Program 
Research,  Development,  Test,  and  Evaluation 


DATSD 

SBCCOM 

JPO-BD 

DARPA 

- > 

◄ . ► 


=  Deputy  Assistant  to  the  Secretary  of  Defense 
=  Soldier  and  Biological  Chemical  Command 
=  Joint  Program  Office  for  Biological  Defense 
=  Defense  Advanced  Research  Projects  Agency 
=  Funding  flows 
=  Informal  coordination 


SBCCOM  is  organized  around  two  integrated  business  areas,  one  of  which 
is  research,  development,  and  acquisition.  Nearly  half  of  SBCCOM 
research,  development  and  acquisition  funding  supports  the  CB  Defense 
Program.  SBCCOM  is  engaged  in  the  full  range  of  research  and 
development  encompassing  both  biological  and  chemical  systems. 
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Chemical  and  Biological  Defense  Program 
Research,  Development,  Test,  and  Evaluation 
Organizations 


SBCCOM  business  areas  include  chemical  detection,  biological  detection, 
decontamination,  protection,  and  supporting  science  and  technology. 

JPO-BD  was  created  in  1994  to  manage  the  biological  warfare  agent 
detection  program.  The  office  monitors  emerging  technologies  for 
advanced  development,  demonstration,  and  upgrades  of  fielded  biological 
detection  systems. 

The  DARPA  Biological  Warfare  Defense  Program  is  an  applied  research 
program  established  by  the  National  Defense  Authorization  Act  for  Fiscal 
Year  1997  (P.L.  103-160,  as  cunended)  to  fund  an  applied  research  program 
supporting  revolutionary  new  approaches  to  biological  warfare  defense. 
The  Biological  Warfare  Defense  Program  pursues  high-risk,  high-potential 
technologies  from  the  demonstration  of  technical  feasibility  through  the 
development  of  prototype  systems. 

DOE’s  CB  Nonproliferation  Program  was  established  in  1997  in  response  to 
the  Defense  Against  Weapons  of  Mass  Destruction  Act  of  1996,  which 
appropriated  $17  million  to  DOE  to  conduct  research  and  development  to 
develop  new  means  for  detecting  the  presence,  transportation,  production, 
and  use  of  weapons  of  mass  destruction  and  related  materials  and 
technologies.  According  to  DOE,  the  purpose  of  this  appropriation  was  to 
ensure  the  full  engagement  of  DOE  national  laboratories  in  responding  to 
the  threat  posed  by  CB  weapons  to  U.S.  civilians.  DOE  funds  research  and 
development,  from  basic  research  to  fieldable  prototypes,  in  pursuit  of 
advanced  technologies  that  can  enable  first  responders  to  more  effectively 
prepare  and  respond  to  the  use  of  CB  agents. 
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The  first  copy  ofteach  GAO  report  md  testimony  is  free. 

Additional  copies  are  $2  each.  Or^rs  should  be  sent  to  the 
following  addres^  accompanied  m  a  check  or  money  order  made 
out  to  the  Superintendent  of  Documents,  when  necessary,  VISA  and 
MasterCard  credincards  are  ac/epted,  also. 


Orders  for  100  or  more  copies!  to  be  mailed  to  a  single  address  are 
discounted  25  perrant.  / 


Orders  by  mail:  \ 

U.S.  General  Accounting J 
P.O.  Box  37050  \  / 

Washington,  DC  20013/ 

or  visit:  jf 


Room  1100  /  \ 

700  4th  St.  NW  (corfaerW  4th  and  G  Sts.  NW) 
U.S.  General  Accounting  Office 
Washington,  DC  /  1 


Orders  may  also  be  placed  bv 
or  by  using  fax  number  (202) 


calling  (202)  512-6000 
512-6061,  or  TDD  (202)  512-2537. 


Each  day,  GAOjissues  a  list  of  newly  available  reports  and 
testimony.  To  receive  facsimile  copies  of  the  daily  list  or  any  list 
from  the  past  BO  days,  please  call  (202)  512-6000  using  a  touchtone 
phone.  A  recorded  menu  wall  provide  information  on  how  to  obtain 
these  lists.  /  I 


For  informmion  on  how  to  access  GAO  reports  on  the  INTERNET, 
send  an  e-mail  message  with  “info”  in  the  body  to: 


info@i 


igao.gov 


or  visit  G^O’s  World  Wide  Web  Home  Page  at: 


http://v 


r.gao.gov 
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